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ABSTRACT 

Wise allocation of resources implies knowledge of 
which actions will yield the highest net benefit. Predicting benefit 
requires comparing objectives as to importance and setting priorities 
among them. This article discusses ways in which policy-makers can 
set priorities rationally. The advantages of ratios for comparing 
priorities are discussed. The value contribution technique for 
estimating priorities is described and illustrated with a set of 
educational objectives. Applications of the method in Asia are 
reviewed^ and the technique is evaluated. (Author) 
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INTRODUCTION 



In any society there are far more objectives worth 
achieving than there are resources to achieve them. It 
makes sense to apply the limited resources available to 
those objectives which are most important, giving due con- 
sideration also to cost effectiveness of action alterna- 
tives. This calls for setting priorities among objectives, 
priorities which reflect the public interest. The problem 
of setting priorities occurs at all levels of government, in 
business, for that matter in nearly any organization where 
decisions are based partly on human values. The issues and 
methods discussed here, though developed for educational 
decisions, are equally applicable to other -settings. 

« 

To set priorities is to make a conscious judgment that 
some objectives, deserve more immediate attention or ^ more 
effort than others. It may be the judgment of one person or 
a social consensus such as a majority vote. The judgment 
may be a direct intuitive decision, or the result of a 
rational analysis. In order to set priorities rationally, 
policy makers must first know what their objectives are. 
Many policy analysts work in an environment of confusion 
over objectives. Their clients are not always sure what 
they want to do and somer 'mes the analyst has to insert his 



own objectives in order to get on with the job. 



Our experience in consulting and conducting workshops 
on objectives, mainly in education, convinces us that the 
whole concept of objectives as a systematic guide to policy 
and action is still rather unfamiliar terrain to most 
policy makers. Means and ends are seldom clearly distinguished. 
Objectives are often stated so generally as to be of little 
use,— "a well rounded life," "a smoothly running organiza- 
tion," "the good society." Because measurement of how well 
an objective is achieved is often difficult, there is a 
tendency to undermine the purpose of stating objectives (to 
make clear what you want) by stating only those things which 
can be measured easily. To boot, the whole idea of specifying 
objectives is often challenged by executives and professionals, 
with the argument that rational analysis in terms of objec- 
"tives fails to account for their complex intuitive achieve- 
ments. 

Such difficulties notwithstanding, the art of specifying 
objectives seems to be gaining momentum in many policy 
arenas. On the other hand, methods of setting priorities 
among objectives are a good deal less advanced. A few 
tentative schemes recently proposed in the field of educa- 
tion are briefly described in a later section, but in 



« 

practice, priorities in all- fields have been set almost 
entirely by intuition. This article explores rational ways 
to set priorities among objectives, where priority is 
determined by the relative benefit expected from achieving 
different objectives. 

There is no escape from the matter of priorities. 
Either priorities are set explicitly or they are implicit in 
the actions and policies adopted. Since implementation and 
corrective actions are generally much more costly than plan- 
ning, there is much to be said for examining priorities in 
advance as a part of planning. If priorities are not set, 
resources many be allocated to whichever needs capture the 
attention first, or by the convenience of the moment, or, as 
is quite common, they may be allocated in the same way they 
have been for years because this does not rock the boat of 
established prerogatives. 



In a field like education where there are typically 
many objectives during a policy decision, if priorities are 
not set explicitly there is a tendency either to ignore some 
objectives or to lean in the direction of giving stated 
objectives equal priority. As an example of the latter, the 
English language curriculum of a school may cite as objec- 
tives "can communicate clearly to a classmate in writing," 
and "can organize written matter effectively, using paragraphs 
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and headings." Alongside these important general objectives 
may appear 20 to 30 objectives dealing with specifics of 
punctuation, grammar and spelling ("uses quotation marks 

correctly,". "capitalizes proper nouns," etc.) . Objectives 
in the area of language mechanics tend to be spelled out in 
great detail because they are easily broken down into 
concrete elements. The danger of not sfi'tting priorities is 
that mechanics may be given a great deal more attention in 
that school than are clear communication or organization of 
writing, simply because mechanics objectives . fill three 
pages and the others less than half a page. When asked • 
explicitly, educators and school boards will usually respond 
that clear communication and organization are far more 
important than any af the mechanics objectives, and perhaps 
more important than all of them combined. But if details of 
English mechanics dominate the content of statements of 
objectives, there is a natural drift toward allocating 
resources mainly to mechanics. 

Since most policymakers have not stated their specific 
objectives the more common problem is identifying exactly 
what it is that policymakers are trying to accomplish. If 
the policymaker is unclear about objectives but quite clear 
about his policy alternatives, these policy alternatives may 
be the main focus of the policy analyst. In this case objectives 
may emerge only as specific benefits and costs in an overall 



study of policy options. 

We favor specifying objective? and their priorities 
separately before analyzing action alternatives. Policy- 
makers have a natural tendency to focus their attention on 
concrete action plans and to look at the consequences only 
secondarily (Campbell and Markle, 1967). A goal for which 
no plan of action comes readily to mind is often ignored 
entirely. If the priorities of goals are determined in 
advance, policymakers and analysts may be better motivated 
to search hard for new ways to achieve those top priority 
goals whic|i„,have been given little attention in the past. 



Quantitative Scales of Pri 



riority 



Many policy analysts are pessimistic about quantifying 
total utility or benefit, especially in the area of social 
benefits and costs (Quade, 1975) . Priority, as a measure of 
•expected benefit, is subject ^to the same problems. We have 
found numeric scaling to be useful in setting priorities, 
despite the uncertainties of measurement, the questionable 
validity of formulas, and the dangers of uncritical deference 
to "number magic" by some policymakers. A skeptical attitude 
by both policymakers and analysts is indeed key to intelligent 
use. Quantification is an aid to thought, not a substitute 
for it. As long as tisers treat it this way, it can be helpful 



to policy analysis. 

Benefit has been quantified on many different scales, * 
Stevens (1951) defines three types of scales for measurement 
along a dimension: ordinal, interval, and ratio, 'The 
ordinal scale puts measured objects in rank order but tells 
nothing of the relative distances between ranks. For 
example, if three educational objectives were ranked first, 
•second and third priority, this would give no indication of 
whether the first had a lot higher priority than the second 
and the second only slightly greater than the third, or if 
the differences were nearly equal. 

An interval scale defines equal intervals on the .scale 
so that differences in priority may be compared using cardinal 
numbers. For example, on t,he scale below. 
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the difference in priority between objectives A and B is 
three times as great as the difference in priority between 
objectives B and C. .However, an interval scrle does, not 
enable one to put priorities in proportion to one another 
and say, for example? that one objective has twice the 
priority of another, 
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A ratio scale corresponds tc the ordinary scale of real 
numbers and .ioes permit meardngful ratios or proportions to 
be expressed. Thus, the achievement of an objective with a 
priority of 8 would be expected to reap twice as much 
benefit as achievement of an objective with a priority of 4, 
And a priority of 0 'would indicate no benefit at all to be 
expected from achievement of the objective, A ratio scale, 
in other words, has a meaningful zero point from which 
distances can be measured and compared in ratios or multiples 

Another unit of benefit is the amount of time a given 
person would give in order to obtain or not lose the benefit. 
Both money and time have different subjective worth to 
different people, of course, so that direct estimates of 
benefit should be. made by the same people if they are to be 
compared, 

• 

Since priority is the same type of dimension as utility, 
a benefit ratio of priority is equivalent to the economists' 
cost-benefit ratio of utility. Some critics of cost-benefit 
ratios (McKean, 1958; Hitch, 1958) prefer to use a cost- 
benefit difference to assess utility. For example, if the 
total estimated benefit of an alternative is $1,000 and its 
cost is $800, then its net utility is the difference, $200, 
Their main objection to ratios is that the unit of benefit 
or cost may actually differ in subjective worth at. different 



extremes of the scale. For example/ a dollar may be of 
greater worth in a $5 transaction than in. a million dollar 
transaction. In such a case ratios would not be equivalent 
throughout the scale. 

However, the same criticism applit?s to differences; if 
the scale is distorted the comparison will be biased. 
Whether the bias is greater with ratios or differences 
depends entirely on the specific niimbers and distortions 
assumed. In the same example just used for ratios, a 
difference of one dollar between benefits and costs may mean 
a great deal more in a $5 transaction than in a million 
dollar transaction . 

The other main criticism of ratios is that they ignore 
the absolute size of the nvimerator and the denominator. 
Thus, a benef it"-cost ratio of 3 to 1 is the same whether it 
is $3 to $1, or $3 million to $1 million. In reality any 
policymaker is aware tha-c much greater stakes are involved 
in some options than in others and takes this into account. 
With a benefit to cost ratio of 3 he will leap at the 
million dollar opportunity and ignore the three-dollar one; 
that is, unless the small one is an estimate of cost-benefit 
per unit and he is contemplating producing 10 million units 
at a cost each of $1 and a return each of $3. And again, 
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the criticism applies to differences as well as to ratios. 
To know that benefit minus cost equals $2 is not enough; one 
must also know whether the transaction is $3 or $3 million.' 
If it is $3 million the $2 difference is trivial. 

Furthermore, cost-benefit differences require that 
costs and benefits be measured in the same units. Often 
this is difficult for social benefits or costs. Ratios on 
the other hand can be compared meaningfully even if the 
numerators have different units from the demoninators . For 
example, we know that 15 apples for $2 is a better deal 
than 20 apples for $5 even though apples and dollars are 

different units, rin ratios, 15/2 is greater than 20/5. All 

i 

that is required. is that the two numerator scales be the 
same and the two denominator scales be the same. 

As long as the policymaker is choosing among alterna- 
tives, and not trying to estimate the absolute payoff of 
each al'-ernative, no standard unit of measure of utility is 
necessary. All that is needed is to know that Alternative A 
is expected to yield 1-1/2 times as much benefit as Alternative 
B and four times as much benefit as Alternative C, for 
example, regardless of what units benefit is measured in. 
This is the typical situation facing public policymakers in 
practice — weighing the relative value of alternative policies 
and outcomes, not estimating their absolute value. 
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They must choose among the available options for 
dealing with a problem, even if all options (including doing 
nothing) lead to an absolute loss of some magnitude. Their 
task in this case is to choose the lesser of evils • If 
benefits and costs are not measured in the same units, cost- 
benefit ratios vbuld* seem to give a clearer basis for choice 
than cost-benefit differences. 

So far, we have argued the merits of ratios for comparing 
total utilities (benefit and costs corr»bined) of alternative 
policies. Let us examine naxt how priority ratios make 
possible utility ratios. 

A decision on how to allocate resources is logically 
based on three main inputs: priorities among objectives and 
unintended benefits, probabilities of success, and costs. 
Probability is a ratio by definition (the expected proportion 
of occasions on which an event occurs); costs are largely 
materials, labor, and capital which are easily expressed in 
monetary terms. Even social costs such as employee stress 
or pollution can often be translated into monetary terms by 
obtaining estimates of the amount of money that people would 
pay to avoid such costs. If priorities too can be compared 
on some ratio scale, the deci j^:\on-maker has sufficient 
information to make clear-cut quantitative estimates of the 
relative benefits of alternative courses of action. 
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To illustrate the advantage of ratio scale of prioritie 
over lower order scales of measure, consider the following 
example: 

Plan A to achieve Objective A has a 90% probability of 
success. and will cost $10 million. 

Plan B to achieve Objective B has a 90% probability of 
success and will cost $9 million. 

Suppose first that we have only an ordinal comparison 
of the priorities of Objectives A and B. If they are of 
equal priority or B is greater, policymaker has enough 
information to decide which plan will yield the higher 
utility. That is. Plan B is expected to yield equal or 
greater benefits at lower costs, so the total utility of 
Plan B is greater. However, if Objective A has greater 
priority than Objective B, the policymaker is stymied, for 
he has no way of knowing whether the difference in priority 
is worth the $1 million difference in cost, or worth only 
$100, or worth $100 million. An interval scale in this case 
provides the policymaker no more information than an ordinal 
scale. 

The ratio scale of priority, however, provides the 
policymaker what he needs in order to calculate utility as 
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the ratio of. benefit to cost. Given that the best estimate 
of the benefit to result from a given plan of action is the 
expected benefit from achieving the plan's objective times 
the probability of success, we can calculate and compare the 
utilities of Plans A and B as shown in the the table below. 



A B 

Priority of objective (Benefit) 40 20 

Probability of success of plan ,90 .90 

Cost ($ million) 10 9 



Utility ' 40 X .90 20 x .90 ^ 

10 9 



The example shown in the table assumes the priority of 
Objective A to be twice that of Objective B (40 vs. 20) . By 
changing these relative priorities to different amounts/ it 
can be seen that the utility of Plan A is greater whenever 
the priority of Objective A is at least 12% greater than 
that of Objective B. It can also be seen from this table 
that the units in which benefit is measured do not matter to 
the comparison of utilities as long as we are not interested 
in the absolute value of the utilities but rather only their 
relative size. That is, we do not care whether the utilities 
of Plans A and B were 3.6 vs. 2.0, or 36 vs. 20 since the 
unit of utility is meaningless. In either case the utility 
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of Plan A is almost twice as great as that of Plan B, Thus, 
the unit of benefit does not matter as long as priorities of 
different objectives are compared as ratios on the same 
scale, whatever it be. 

The example above shows the advantage of a tatio scale 

of priority over any lower order scale, in that it enables 

policymakers to compare utilities of different plans of 

action also on a ratio scale. To know that one plan has 

seven times as great a utility as another, for example, is 

more useful to a leader who must allocate resources than 

merely to knoy? that one plan has greater utility than 

another T If the difference in utility were only 1%, then 

even though one plan had slightly greater utility, other 

uncalculated factors such as availability of personnel and ^ 

« 

resources would outweigh this slight difference in deciding 
which plan to choose. But if one has seven times the 
utility of the other, such factors of convenience would 
likely be put aside in order to chcose the plan of 
greater utility. 

The advantages of a ratio scale of measurement of 
priority become greater as the number of plans and objec- 
tives increases, for typically it will be unusual for a 
single plan to have both the greatest benefit and the;; least 
expected cost, the only situation in which ordinal and 
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interval scales clearly indicate one choice. 

Of course, the advantages of a ratio scale of priority 
hold true only if the user finds the scale meaningful and 
can make reasonably stable, consistent estimates. Most 
policymakers • are familiar with ratio scales such as percentage, 
money and time, so that an adequate ratio scale is often 
feasible. 

Dirgct Judgments of Priority 

A person judging the priority of a particular objective 
may m^Vp. a direct judgment on the dimension of priority 
itself. Or he may analyze the objective into a niomber of 
specific consequences and other related factors, evaluate 
these items separately, and then somehow combine them into 
an overall judgment of priority. The latter procedure is 
sometimes called decomposition, or disaggregation. 

Several types of ratio scales have been used to get 
direct judgment of priority. One type is provided by 

answers to the question, "What would it be worth (in money) 
to achieve this objective?" or "What is the most the com- 
munity should pay. to achieve this objective?" Another 
approach is to assume a fixed quantity of resources (e.g., 
$10 million) to distribute among the various objectives. 
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The procedure lends itself to the use of mechanical aids 
such as washers, poker chips, or magnetic tape, which can be 
divided into piles or segments of various size representing 
different allocations. Such aids permit quick review of 
allocations by v.-f.sual scanning so that adjustments can be 
made guic.cly. Techniques of this kind have been- explored by 
other investigators (e.g. Webb, 1972, and Peterson, : 1972) as 
well as by the -authors. 

The main advantages of direct judgment are its speed 
and economy. The main weakness is that the reasons or 
mental steps by which a priority is derived are not easily 
retraceable and there is no systematic accounting of specific 
factors, relevant to. the, judgment. It could be argued that 
this presents no problem as long as different users show 
high agreement on the independent ratings of priority in the 
same policy situation. The assumption is that, whatever the 
reasons, if agreement is high there will also be high agree- 
ment on the implications of the priorities for policy. A 
counter argument is that the priority rating may be sensi- 
tive to the particular guidelines or the way in which the 
objectives are stated (Stake and Cooler, 1970). Thus, 
changing a few. key words with evaluative connotations might 
greatly alter the perceived priorities of all raters. There 
is some support for this concern in the finding of many 
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investigators (Fischer, 1912; Hammond/ 1971, Huber, et al., 
1969) that intuitive judgments tend to focus on very few 
dimensions, regardless of the decision-maker's intent to 
take many factors into account. A decomposition procedure 
which forces the person to examine each dimension and 
consequence separately may correct such errors, if the 
specific factors reviewed include those most important to 
the decision. 

Decomposition Techniques 

When judgment of the priority of an objective is 
decomposed into specific factors, the factors may be of many 
different kinds. For example, the priority of arithmetic 
skills as an educational objective can be analyzed in terms 
of the consequences of having those skills, such as providing 
for family more economically or success in a business 
enterprise.' If each of these consequences has its own value 
more or less independently of what other consequences are 
achieved, then it makes sense to add the expected values of 
these consequences together to obtain an overall expected 
value- of achieving the objective. 

Another way of decomposing priority is to list the 
conditions which are jointly necessary for any value to be 
realized. For example, if arithmetic skills are to have' 
high priority for an adult education agency, then the following 
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factors must occur jointly: 

1. There are positively valued consequences (e.g., 
success in business) which arithmetic skills are 
likely to help one achieve. 

2. People do not already have the needed arithmetic 
skills. 

3. Imparting these skills falls within the agency's 
realm of responsiblity. 

Perhaps other conditions could be named as well, but 
the point is that each factor creates priority only to the 
extent the other two factors are also present. For example, 
if any of these three conditions is totally absent the 
priority of the objective would logically be nil for that 
agency. It is generally accepted that factors which interact 
in this way should be multiplied together to obtain an 
overall priority rating. 

The simple example above illustrates the main stages of 
any decomposition procedure for setting priorities: deciding 
what components or factors the priority setters should 
consider? deciding what type of judgment should be made 
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about each factor; and deciding how to re-combine the 
judgments of specific components into an overall priority 
rating. 



Many studies have compared different mathematical ways 
of combining factor judgments into an overall evaluative 
rating (Huber,et al., 1971, Fischer, 1972). Results Vary 
somewhat according to the type of breakdown but in general 
the results indicate that the mathematical method of combining 
matters little to the overall result, as long as one includes 
only methods that do not violate common sense. The final 
set of priorities or evaluations obtained correlate rather 
highly among nearly all such methods. The technique to be 
recommended here uses the simplest logical combination of 
rules, which turns out to be addition or multiplication 
depending upon the logic of the variables involved. But 
first, let us briefly review specific decomposition procedures 
developed by others. 

Robert Stake (1972) developed a fairly simple procedure 
for use by school teachers and administrators who wish to 
set priorities among objectives. The basic philosophy of 
Stake's technique is that there are three or four types of 
factors which priority-setters should consider carefully, 
but that the way in which these factors are combined should 
be left to the intuition of each priority-setter. The 
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factors to be considered for each objective listed are arranged 
in a matrix and include: the need for achievement of the ob- 
jective as seen by the teacher, the learner, and the community; 
x^yhat resources would be allocated to the achievement of the ob- 
jective; the. probability that a specified allocation of resources 
would achieve the educational objective at a certain level; and 
contingency conditions which should be considered in the in- - 
structional process such as what objectives are prerequisite 
to others. Stake's process of priority planning is quite broad 
and includes vzhat we have defined as the total utility assess- 
ment, including allocation of resources. 

There are many other variations pf the matrix format in 
which objectives, or other outcomes are listed on one dimension 
and slcarnatives or factors whicli affect those outcomes are 
listed on the other dimension (Brown et al., 1974; Goeller, 
1974) . In each cell of this matrix is entered a quantitative 
or verbal estimate of the impact of that factor on that out- 
come. Quade (1975) reports much better reception by policy- 
makers to scorecards or matrices which quantify only specific 
benefits and leave all the aggregation to the judgment of the 
policymaker. Other ways of setting priorities without quantify- 
ing the value of objectives on a numeric scale are discussed 
by Ellon (1972) and include: ranking objectives in order of 
preference and then optimizing them in sequence; and choosing 
only the most important goal while 



setting minimum acceptable levels of attainment for all 
other goals. Churchman et al. (1957) illustrate a procedure 
for weighting "intangible" objectives mathematially when the 
weight of tangible economic objectives is known. 

Two techniques have come -to our attention that used 
matrices to decompose priority and aggregated the elements 
mathematically to get an overall numeric index of priority. 
Cetron (1971) has described a "cross-support matrix" tech- 
nique for priority setting in education. His matrix related 
educational and academic disciplies, such as agricultural 
science^ economics and mathematics, to national goals such 
as agricultural productions, erliination and public works. By 
weighting the value of each goal and estimating the cross 
support among disciplines and goals, he obtains a "total 
relevance" score for each discipline as a measure of its 
priority. In the process he uses a quasi-logarithmic scale 
and transforms the results in a rather complicated and 
dubious manner, where a simpler linear scale might have 
served better. 

The "relevance tree" technique is a similar matrix 
approach (Heneveld, 1970). It assumes two levels of objectives, 
the lower level objectives being related to the more general 
higher level ones. The technique is open-ended in that the 
user is asked to choose his own criteria of importance, such 
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as economic urgency and cultural feasibliity. Each ob- 
jective is then rated on 'a numerical scale for each criterion 
of importance. These ratings are added together and com- 
bined with oth?r factors (amount of change desired; relation- 
ship between lower and higher level objective) to obtain a 
final rating of priority for (>ach lower level objective. 



THE VALUE CONTRIBUTION (VC) TECHNIQUE 
FOR SETTING PRIORITIES 

We developed the value -contribution technique as part 
of an educational planning project in Indonesia^ (Nichols^ 
1972) which included setting priorities among objectives. 
Existing technique? seemed inadequate^ so we invented a new 
method which used ratio scales and spelled out the elements 
going into a judgment of priority so that they could be re- 
examined and justified later.' 

We called it the value-contribution technique because 

the basic kind of judgment on which it is built is estimation 

of the relative proportions. contributed by various sources 

to some valued achievement. In staff tryouts, judging 

relative contributions to the total value of some achievement 

« 

seemed more meaningful to staff than other types of value 
judgments on a ratio scale. 

The fundamental formula in the VC method is P =V-D^ 
meaning that priority (P) of an objective is the product of 
its value when fully achieved (V) times the discrepancy 
between current and desired levels of achievement (D) . Why 
do we multiply V and D rather than^ say^ adding them? 
Because an objective has priority only to the extent both V 
and D are jointly present. If either is zero no benefit can 
be expected. That is^ if the value is zero the priority 



should be zero because the achievement has no value, if the 
discrepancy between ideal and current level of achievement 
is zero, then the objective is already achieved and the 
priority should be zero since no further improvement is 
expected. Mo^re generally, priority is directly proportional 
to value if D is held constant; any multiplication of value 
multiplies priority in the same ratio. The same holds true 
for discrepancy (D) if value is held constant. These assertions 
are based on the definitions of the variables, not on empirical 
data. 

The value of an objective (V) is determined by its 
contribution to higher-order goals, each contribution being 
weighted by the value of the goal itself. Thus, if the 
objective, "can read," contributes to only one higher goal, 
the value of the objective is 

^ ^ '^I'^l' '^l the contribution of reading 

ability to Goal 1, and is the value of 
Goal 1. 

Note again that v-i multiply the two factors because if 
either or is zero, the objective has no value in 
relation to that goal. 

If the objective contributes to two independent goals, 
its value is V = (C^-V^) + {C^'V^) 

For example, consider the goals: 
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Goal* 1 = Has skills needed for usefvil, rewarding 
work. 

Goal 2 = Enjoys diverse recreational pursuits. 

Assume Goal 1 is four times as valuable as Goal 2, so 
= 4 and V2 = 1 

Now let us assume these two goals are the only ones 
which the objectives "can read" and "can sing" contribute 
to, which is clearly not true but serves to keep the example 
simple. If reading contributes 9 times as much to Goal 1 as 
singincT does, then 

^rl = 9 and C^^ = 1 

If reading and singing contribute equally to Goal 2, 
then = Cs2 = 5 

(The contributions of all objectives must sttm to the same 
total for every goal if C and V are to be independent; in 
this example the arbitrary total is 10.) 

From the above, we calculate the value of reading to be 
\ " ^^rl*'^^ ^^r2"^2^ " ^ 4) + (5 X 1) = 41 

and the value of singing to be 

^s " ^^sl'^1^ ^^s2*^2^ = (1 X 4) + (5 X 1) = 9 
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Using the two goals in the above example makes it clear that 
values calculated for objectives will be good estimates only 
to the extent all goals served by those objective's are taken 
into account. Therefore it is important that the set of 
goals be comprehensive. However, knowing that human priority 
setters will never in actuality list every relevant goal and 
consequence of value, the VC method allows for a correction, 
factor called R, which is the residual value of an objective 
beyond its contribution to stated goals. In arithmetic 
terms, R is the proportion by which V should be increased 
because of the objective's residual value. Thus if reading 
were judged to contribute to other goals besides #1 and #2 
above, and this residual value amounted to 50% of its value 
in service to Goals 1 and 2 cc-riincd, then R would bs .5 and 
the total value of the reading objective would be 

= 41 X (1 + R) = 41 X 1.5 = 61.5 . 
In practice such a large value of R should suggest to 
priority setters that important goals have been left unstated 
and should be identified and added to the set of explicit 
goals. In applications by the authors to date the goals have 
been quite comprehensive, so the values of R for educational 
objectives have usually been zero and in no case greater 
than .02. The effect of R on priorities in such cases is 
negligible. 

One other type of factor should be included in the 
final formula for calculating priorities, and that is limitations 



of ability to achieve the goals and objectives. In the case 
^of a goal this means the extent to which achieving all the 
listed objectives is sufficient to achieve the goal. 

For example, the goal of economic well-being for every 
person depends partly on being able to read -and achievement 
of other educational objectives, but it also depends on 
health, family wealth and the local economy. If these other 
factors combined account for 40% of what it takes to achieve 
the goal, then only 60% can possibly be achieved by mastery 
of the stated educational objectives. Therefore in the 
priority equation the value contribution of all objectives 
to that goal should be reduced to .60 of C«V. If we call 
this "ability limit" factor "A," then the value of any 
objective (V ) would be limited as follows': 

f^al-^r^l) + ^^^2*^2-^2^ ^ ^^aa'^a-^a^ ••••etc. 
In the above example, if economic well-being of a person is 
Goal 3, then 

^a3*^3*^3 = ^aa-^a-^-^) 

This completes the basic calculations of general priority 
of objective a, and we have 

P = V 'D 

a a a 

Thus, if the value of the objective "can read" (V^) , is 41 
(from an earlier example), and we wish 30% more youngsters 
to achieve It than now do (D^ = .30), then the priority is 



= 20 X .30 = 6 
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Up to this point we have calculated the priority of an 
objective from the total community's viewpoint. But priorities 
for an agency withii> the community, such as schools, may be 
different because their responsibilities and capabilities 
are specialized and limited. E'or example, if learning to 
read is 90% within the ability and responsibility of the 
schools to achieve, then in calculating school priorities 
(as opposed to community priorities) the result should be 
reduced to 90%. Thus, the educational priority of the 
reading objective, using the numbers in the prior example/ 
is 

EP^ = 20 X .30 X .90 = 5.4 
Or for any objective a, 
a 'a "a **a 

We use the same letter, a, as used earlier in denoting the 
limitation of ability to achieve a goal, because again the 
factor is a limitation of ability— in this case the ability 
of the agency to achieve the objective. 

Summary. To summarize the above logic, the value of an 
objective (V^) is estimated from its expected contribution 
to goals having different value., 

= (1 + R^) [C^i-Vj^-A3^) + {C^2*V2-A2) + ...etc.], that is, 

•^a = + ^a^-f^^agVg^' "'^^^'^ "^^^ ^um across "■: 

all goals. " 



29 



The community •priority of an objective is its value times 
the discrepancy between desired and current levels of 



achievement, P = V .D , 
a a a 



The priority of the objective for a given agency is the 
community priority reduced by the agency's limitation of 
ability to achieve the objective. If the agency iW the 
schools, then educational priority is 

^^a = ^a-^a-^a' 

EP = (1 + R )X(C^ V A )-D -A 
a a g ag g g' a a 

All factors on the right-hand side of the formula above are 
subjective judgments which may or miy not be more valid and 
reliable then direct judgments of priority. But they do 
combine what would seem to be the basic ingredients of 
priority in a rational manner. Others who study the relation, 
of priority-setting procedures to decision quality may well 
improve upon the above formula • From our perspective of the 
moment, it seems to be the most logical formula. 

Illustrative Calculation of Priorities by the VC Method 

The fcTlowing example is a hypothetical case of one person 
setting priorities among 12 objectives which serve five 
community goals. 

The community goals and their relative values are: 



Judged 

Value Goal * 

10 !• (Economic livelihood) All adults have sufficient 

income to live in moderate comfort. 

8 2. (Self-realization) Each person has the 

opportunity and encouragement to realize 
fully his own potential as a human being. 

6 3. (Social Harmony) There is social harmony among 

all groups and individuals most of the time. 

5 4. (Nature) The natural environment of the community 

is pleasant/ healthful and well preserved. 

6 5. (Government) The governments which serve the 

community are efficient and responsive to all 
citizen* s needs. 



The goal values were derived by assigning the most valued 
goal an arbitrary value of 10. Each other goal was then 
given a value proportionate to the one valued at 10. As a 
check, the other 4 were compared with each other and^ the 
values adjusted until all pairs seemed to be in approximately 
the correct ratio of value. If the reader disagrees with 
these values or- any of the other judgments made in this 
illustration, it may be worthwhile to recalculate values and 
priorities substituting his own judgments in order to get a 
sense of how the results vary according to such differences 
in judgment. 

Suppose the school system in this community has adopted the 
following major educational objectives for its students, and 
wishes to set priorities among them so that it may be better 
prepared to plan and allocate resources: 
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Objective • . 

a. " Can read, write, listen and speak effectively in his 

native language. 

b. Can communicate in a foreign language. 

°' learning''^'"^^ skills of study and inquiry, and enjoys 

d. Has acquired arithmetic skills and key concepts in 
mathematics i - 

e. Can effectively plan and manage his own time and resources, 
or those of a group. 

f. Participates effectively as a citizen; contributes to 
community welfare. 

g. Treats other people humanely and ethically; keeps commitments 

h. Develops own values and uses them to critically evaluate. 

i. Appreciates humanity's cultural diversity and the common 
characteristics of human beings. 

j. Cultivates expressive communication and appreciation in 
the arts, music and/or literature. 

k. Understands the physical v/orld and man's relations to it. - 
1. Maintains good physical and mental health. 

The estimted contribution of each ojbective to each goal is 
shown in Table 1. The initial procedure used for each goal 
was to pick a highly contributing objective and arbitrarily 
call its contribution C = 10, then judge the other objectives- 
contributions in proportion: For example, being able to use 
one's own language well seems quite important to having a 
job, so it was assigned a C of 10 for Goal 1. The estimated 
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contributions -(Est. C) to each goal are shown in the l^t^ 
hand column under each goal in Table 1. ( i 

The estimated C 'for each objective under a goal used an 
arbitrary reference value of 10 for convenience. The logic 
of the method requires that C sum to the same number for every 
goal. To achieve this the estimates of C were adjusted by 
a constant for each goal. The constant is computed for each 
goal at the bottom of Table 1, and the adjusted estimates of 
C in the "Total" row differ slightly from 60.0 only because 
of rounding error. All calculations in this illustration 
are rounded to 2 or 3 digits because the estimates are assumed 
to be no more precise than this. 



Table 1. Contribution (C) of each objective to each goal 



Goal I 



Goal 2 



Goal 3 



Goal 4 



Goal 5 



Short title 
for obi active 


Economic 
livelihood 
.Est C Adjus C 


Self- 
realization 
. Est C Adjus C 


Est 


Social 
harmony 
C Adjus C 


Est 


Nature 

C Adjus C 


Government 
Est C Adius 


a. Own language 


10 


8*. 6 


8 


5.9 


4 


4-0 


3 


2.9 


8 


5.8 


b. Foreign 

language 


1 


0.9 


5 


3.7 


4 


4.0 


1 


1.0 


2 


1.5 


c. Inquiry/ 

learning 


8 


6.9 


10 


7.4 


4 


4.0 


6 


5.7 


10 


7.3 


d. Math 


6 


5.2 


2 


1.5 


0 


0.0 


1 


. ?..o 


5 


3.7 


e» Plan and 
manage 


n 


9.5 


10 


7.4 


3 


3.0 


6 


5.7 


7 


5.1 


f» Citizenship 


2 


1.7 


5 


3.7 


8 


8.0 


12 


11.4 


14 


'10.2 


g. Treat others 
veil 


7 


6.0 


3 


2.2 


10 


10.0 


5 


4.8 


9 


6.6 


h. Own values 


3 


2.6 


9 


6.7 


5 


5.0 


2 


1.9 


8 


5.8 


1. Humanity 


3 


2.6 


6 


4.4 


9 


9.0 


4 


3.8 


8 


5.8 


J. Arts 


2 


1.7 


9* 


6.7 


5 


5.0 


1 


1.0 


1 * 


0.7 


k. Physical 
world 


5 


4.3 


4 


3.0 


3 


3.0 


• 12 


11.4 


6 


4.4 


1. Health 

Total 


12 


10.3 

6U. 3 


10 7.4 
81 60.0 


_5 
' 60 


5.0 
60.0 


10. 
63 


9.5 
60.1 


_4 
82 


2,9 
59.8 


Adjustment 
Factor 


60 
70 


- .86 


60 
81 


- .74 


60 
60 


" 1 


60 
63 


« .95 


60 
82 


.73 
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Next we estimate the extent to which each goal can be fully 
•achieved by achievement of the 12 stated educational objectives. 
This judged limitation (A) is showni f or each goal in the first 
column of Table 2. For example/ "economic livelihood" is 
judged to be only 60% achievable through these 12 objectives 
above, while "self-realization" is judged to be 9.5% achi^yable 
through these 12 objectives. 



For each objective the product of V and A (that is, V-A) is 
then multiplied by the contribution (C) of the objective to 
that goal. The right-hand side of Table 2 illustrates these 
calculations for one objective (a. own language) . The con- 
tribution of Objective a for all 5 goals together is the sum 
of the CVA, which is S (CVA) =156. 

Table 2. Ability limits (A) and values (V) of each goal 
and calculation of the value of Objective a. 





Goal 


A 


V 


V-A 


X 


Ca 


= CVA 


1. 


Economic livelihood 


.60 


10 


6.0 


X 


8.6 


= 52 


2. 


Self-realization 


.95 


8 


7.6 


X 


5.9 


= 45 


3. 


Social harmony 


.85 


6 


5.1 


X 


4.0 


20 


4. 


Nature . 


.7j 


5 


3.8 


X 


2.9 


11 


5. 


Government 


.80 


6 


4.8 


X 


5.8 


28 



Total, or E(CVA)= 156 
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By the same formula, H (CVA) has been calculated for each of 
the .J. 2 objectives and the answers are shown in the first 
column of Table 3. The remainder of Table 3 shows the final 
calculation of the educational priority of each objective. 
In preparation for this calculation the residual values (R) 
of each objective were estimated. If "plain unconstructive 
fun" had been included as a goal, most of the residual values 
would have been much smaller. In the case of "math" the R 
of .25 is attributed mainly to the practical convenience of 
math around home, shopping, banking, etc., which are not included 
in the 5 stated goals . 

The column to the right of R is the total value (V*) of 

the objective, including the R factor. For example, the value 

of objective a was increased by 15% from 156 to 179 because 

« 

R = .15. • The next column is D, the discrepancy between - - 

desired and current actual proportion of youth achieveing the 
objective. For objective b the desired level was .50, meaning 
half the graduates should know a foreign language, and the current 
level .10 yielding a D of. 40. For objective j_ (arts) the 
desired level was .90 and the current level .40 yielding a D 
of .50. For all other objectives the desired level was set 
at 1.00, meaning all youth should achieve it, and the D shown 
is the difference between 1.00 and the proportion estimated 
to be achieving the objective currently. 
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The next columK in Table 3, labeled A^, is the factor reflecting 
-.the limitation of the schools' ability to biring about full 
achievement of each educational objective. These judgments 
reflect the schools' responsibility ai\d ,the state of the art 
of teaching, but not current flaws in the local schools which 
could be corrected by appropriate action within a reasonable 
time. is near l-.O for math (Objective d) because the 

schools have the responsibility and ability to achieve it. 
On the other hand A^ is only .30 for "Treat others well" 
(g) because that objective is judged to depend mostly on 
factors outside school . 

• • 

Finally, educational priority (EP) is the product of V, D and 
A^. PriOo-lLy for the community is simply P = V-D. as noted 
earlier. But to obtain educational priority we must raultiply 
priority by the school's ability to achieve each objective. 

Thus* EP = V-D-A . 

s 

Table 3. Calculation of educational priorities (EP) of objectives 

Objective 

a. Own language 

b. Foreign language 

c. Inquiry/learning 

d. Math 

e. Plan/manage 

f. Citizenship 

g. Treat others well 

h. Own values 

i. Mankind 
j. Arts 

k; Physical world 

1. Health 

* Total value of an objective, V = (1 + R) r (CVA) 



E(CVA) 


R 


V* 


'- D 


A 
s 


EP 


156 


.15 


179 


.35 


.85 


. 53 


65 


.20 


78 


.40 


.•95 


30 


175 


.15 


201 


.70 


.80 


113 


64 


.25 


80 


.jO 


.95 


38 


175 


.20" 


210 


.55 


.60 


69 


171 


.05 


180 ' 


.80 


.60 


86 


154 


.20 


185 


.30 


.30 


17 


127 


.20 


152 


.40 


.50 


30 


137 


.20 


164 


.45 


.75 


55 


. 94 


.20 


113 


.50 


. 6y) 


34 


128 


.10 


141 


.30 


.7.;. 


32 


194 


.35 


262 


.50 


. .4C 


52 
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The superintendent of a school district who arrived at the 
•above priorities might find some surprises. Inquiry techniques, 
love of learning and citizen skills are frequently paid lip 
service, but to find that tlieir priorities for action are • 
from 1 1/2 to 3 times as great as nearly all other objectives 
puts them in a new light. It might lead the school district 
to search harder for ways to achieve these objectives and 
perhaps to invest more of the school dollar in them. 

Priority-setting by a q^^.^p, Setting priorities for a 
community or society is a task seldom delegated explicitly 
to one person, though one person often controls the process 
temporarily by default. More often a group, of elected 

and/or appointed officials determir.cc pricriticc — ' ••■>" 

does so implicitly through its ...action decisions rather than 
by setting priorities as a disti^nct task in itself. For 
policy-making groups undertaking the explicit task of setting 
priorities, methods of achieving consensus must be considered. 

Sen (1970) has discussed the problems of aggregating personal, 
utilities into a collective utility, which is equivalent to 
aggregating priorities. Arrow (1963) concludes that utilities 
cannot be compared between persons, but does so apparently 
on theoretical and philosophical grounds. In a more practical 
vein. Van de Ven and Delbecq (1972) have recently summarized 
the evidence concerning what types of decisions are best 
suited to different types of group processes. They distinguish 



between interacting groups and nominal groups, in which 
members do not interact with each other. On the basis of 
the available evidence, they recommenc-. that nominal group 
processes are better for fact-finding and idea generation. 
For a number of reasons an interacting group inhibits many 
of the members and suppresses creative thinking. Nominal 
groups, in which individuals work alone, tend to excel over 
interacting groups (including brain-storming, groups) in the 
quality, quantity, and variety of ideas produced. They also 
suggest that time may be used more economically in nominal 
groups since tasks can be started and stopped more quickly. 

When the task of a group is to synthesize information or 
work tovrarct consensus in evaluation, the research suggests 
that interacting group processes are at least as effective 
as nominal group processes. It would appear that priority- 
setting emphasizes the tasks of synthesis and reaching 
consensus more than, it does creative generation of ideas. 
On this basis either interacting or nominal groups or some 
conbination might be appropriate to the task of setting 
priorities. 

However, there is a substantial body of evidence confirming 
that the judgments of individuals are strongly influenced by 
the judgments of other members of the same interacting 
group. In view of this it would seem sensible to obtain 
independent judgments from the members of a priority-setting 
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^7 

group first, so that the initial range of disagreement could 
be estimated accurately. Afterwards, group discussion might 
be used as a basis for reaching consensus. This is the 
procedure recommended by Huber and Delbecq (1971) lor practicing 
managers of decision conferences, and is the principal which 
underlies some uses of the Delphi technique (Helmer, 1966). 
Most applications of the Delphi technique involve' repeated 
cycles of individual judgments in nominal groups, with 
summary information about judgments and reasons displayed 
anonymously to each person. 

If a group planning to use the VC method is not too large we 
recommend that a modification of Delphi technique be used to 
arrive at a single group estimate of each factor. That is, 
each member begins by independently making his own estimate 
of the factors from whatever evidence is on hand and his own 
experience ^nd values. After these are recorded the group 
accepts tfhe group average (mean) if there is close agreement 
on a factor, but discusses reasons for their judgments 
wherever there are sizable discrepancies. If the discussion 
yields a clear consensus, this is accepted as the group 
estimate. If consensus is not clear, another record of 
independent individual judgments may be taken and again 
reasons for discrepancies discussed. If substantial disagreements 
persist, a group average is accepted and the more discrepant 
individual estimates are appended to the record so that the 
sensitivity of the results to these differences may be 
calculated. 



er|c 39 



The size and composition of groups appropriate for priority- 
setting must depend in part upon the range of knowledge and 
expertise required to make the individuda.l judgments competently. 
Huber and Delbecq suggest that, in general, adding members 
beyond the group size of 10 seems to contribute little to 
the reduction of judgmental error. Large interacting groups 
also tend to take longer to complete a given task and represent 
larger expenditures of man-liours of effort. If larger 
groups are needed in order to represent the full range of 
expertise needed, it may be better to divide the task into 
subgroups or committees approximately 10 members in size, 
with a coordinating committee to combine the work of the 
various subgroups. 

Since the time which skilled personnel have available to set 
priorities is limited, it is important that their time* be 
focused on those parts of the procedure which have the 
greatest impact on the final priorities derived. The parts 
having greatest impact are defined as those steps in which 
variations in human judgment make the greatest difference in 
the numerical priorities which result. Estimating contributions 
of objectives to goals takes the most time and each separate 
judgment has the least impact, so it may be hastened by' 
delegating the task to smaller subgroups or individuals'. To 
the extent that different members of any group are differentially 
knowledgeable about different goels or objectives, the task 
can be divided among subgroups so that each member concentrates 
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on those areas where his expertise is greatest, when time 
^is critical the task can also be speeded up. by allowing 
wider limits of disagreement for accepting a group consensus 
estimate without further discussion. Also, most of the 
arithmetic calculations in the VC process can be completed 
by a clerical assistant while priority setters are completing 
their estimates. 

When a higher level group is reviewing the work of a sub- 
committee staff group, time can be saved if each reviewer 
first independently reviews the values estimated by the 
staff and circles any values that he disagrees with by a 
substantial margin (say 20%). if a clerk tallies a list of 
the items circled by one or more reviewers, discussion can 
then be limited to those, few items. Twice in the Indonesian 
application higher level government officials reviewed staff 
estimates, and in both cases the number of changes resulting 
from a fairly thorough review was very few. Although there 
may be cultural differences, this suggests that adding the 
review process may not change the final priorities much. 

Beyond small face-to-face groups, many studies of objectives, 
needs, and priorities have involved collecting judgmental 
data from larger samples of people from appropriately defined 
populations. Stake (1970) has reviewed the status of this 
research. In many educational needs assessments in the 
United States in recent years samples of students, parents. 
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educators, and* other citizens have been asked to rate a list 
of objectives or needs on some type of scale. Typically 
these individual judgments have then been combined statis- 
tically to obtain an index of priority. 

Another approach^ using the critical incident technique 

(Abbott et al.r 1968)^ involved collecting thousands of 

specific accounts of incidents which demonstrated effective 

or ineffective education of youth in a particular school 

distirct. These incidents were then categorized into community 

concerns as a basis for later development of instructional 

objectives. It is tempting to define educational priority by 

the number of citizens who mentioned incidents in a particular 

category of concern^ but this is probably not sound. The 

number of behaviors reported in a category of concern may 

well reflect the salience of this category in the public 

mindf but the above study suggests that it does not reflect 

perceived importance or educational priority; a sample of 
citizens rated the importance of the categories and the 

product-moment correlation between salience (number of 

incidents) and rated importance was only +.16, indicating at 

most a weak relationship between salience and rated importance. 

Applications of VC and other Ratio Techniques 

The initial application of the value-contribution 
technique in Indonesia is described in earlier reports 
(Nichols, 1972; Campbell, 1974). Subsequent applications in 



the Philippines are described by Jasin (1973). The Indonesian 
and Philippine reports both detail specific problems and 
recommended solutions in setting priorities. The following 
comments were made at a Southeast Asian Conference on primary 
education by Sudijarto (1974), a high ranking educational 
official in Indonesia who was mainly responsible for our 
priority setting efforts there. ' 

"Before the first Five-Year Development Plan, politicians 
wanted 1% to 1 1/2% of the population to complete university, 
and the result was that almost all money was put into the 
universities. It was not until we started using some of 
these priority-setting techniques that we realized how 
poorly v;c v/ere using our resources. Beginning with the 
first Five-Year Development Plan we started to allocate our 
resources differently." 

« 

In the Philippines Jasin compared empirically two variations 
of the value contribution method of setting priorities with 
the method of estimating priority ratios directly. They 
streamlined the VC technique and reduced calculations to the 
key variable "value." The four techniques of computing 
value ratios that they tried correlated quite highly (pp.rt- 
whole correlations of .76 to .88), even though entirely 
different groups of persons used the different techniques. 
This gaye them more confidence in the overall approach of ■ 
usingj^atio judgments. They concluded that although simple, 



direct estimates of value were the quickest, the safest 
.procedure was to use a relatively simple variation of the 
value-contribution technique so that the basis for judging 
value could be analyzed and retraced, and those making the 
judgments would have to focus their attention on specific 
goals and on ways objectives contributed to these goals. 
The reports above also -include a number of practical recom- 
mendations on training in use of numerical scales, number of 
persons in a judgment group, and number of specific objectives 
within a more general objective. 

Another factor in priorities which has been dealt with 
in various ways in different applications is the extent to 
which an objective can be achieved through formal schooling. 
The ability of schools or any other institution to achieve 
an objective has been built into our earlier formulas for 
priority as a quantitative factor. Jasin has dealt with it 
as a separate qualitative consideration after value was 
calculated. This has some real advantages. The main one is 
that priority should be considered first of all as a general 
matter for the whole society. if each agency, such as 
schools, religious institutions, law enforcement, and other 
agencies, calculates its own priorities separately, many 
important objectives may be omitted from the high priority 
list simply because no one agency clearly feels it has 
responsibility for achieving that objective. Therefore it is 
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probably best to consider priority as a general matter and 
decide afterwards which agencies and institutions should 
cooperate to achieve the objectives. 



CRITIQUE 

The art of setting priorities is clearly still in its 
infancy. Our techniques are tentative, and the few analysts 
who deal with priorities quantitatively do so experimentally, 
changing this or that feature to see what works best. 
-Different problems call for different analytic structures , 
of course, and clients vary greatly in attitudes toward 
decomposition and quantification. 

The main obstacle to wider application of decomposition 
techniques has been their complexity. As long as rapid 
intuitive judgments on priorities are accepted uncritically, 
policymakers will tend to follow this more expedient and 
economical habit. The value-contribution method of setting 
priorities is as simple as we could make it without ignoring 
vital realities. Everything but estimating the factors is 
easily automated. Still, the number of human estimates 
required may seem too large an effort to most policymakers 
until there is substantial pressure for specific justification 
of decisions and priorities. 

One of the advantages of the techniques such as VC is 
that the many steps by which priorities are derived are 
explicit and retraceable, so that those who disagree with a 
given priority and wish to locate the specific judgments 
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which account for the disputed priority can do so. In this 
way public policymakers can justify their decisions by 
making public the detailed set of steps and judgments on 
which a decision was based. This can add appreciably to the 
trust among various levels of public authorities and to the 
credibility of public interest as the primary consideration 
in public decisions. 

The applications of the VC technique described here 
assume public interest to be the total value base. The 
technique could just as easily include other values such as 
profit or power, which would be more appropriate for busines 
and other competitive enterprises than for a government 
which exTRts only to serve its people. Of course other 
motives such as personal gain often do influence public 
decisions, and those who wish to predict or describe the 
actual behavior of decision-makers probably improve their 
accuracy by taking such motives into account. But if the 
intent is to set priorities in a way that best serves the 
public interest/ then it is appropriate to consider only 
public goals. This does not deny that public authorities 
have personal motives as well. It means only that they are 
utlimately . accountable to the public. And with an ever more 
alert citizenry leaders will likely be called on to justify 
their priorities in terms of the public interest, no matter 
how they set priorities personally. 
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The main source of variability or unreliability in 
•setting priorities by the VC and related methods may lie not " 
in the calculations nor in the estimates themselves, but in 
the prior task of specifying objectives and goals. If 
important objectives or goals are omitted and donft come to 
mind in estimating the priorities may vary appreciably. 
Conversely if two goals overlap so that some of the same 
achievements are included in both, the resulting priorities 
will be biased in the direction of overestimating the importance 
of those achievements. These weaknesses in goal and objective 
statements can be minimized by weeding out redundancy at the 
start, and by a thorough review of the statements for important 
omissions. 

The priorities derived by the vc technique are numerical 
estimates of expected benefit and can be compared as ratios. 
As noted earlier this means that cost-benefit ratios can be 
calculated and compared for different policy alternatives. 
The rational decision maker can thereby arrive at a clear 
decision to the extent he trusts the method and the inputs. 
If the. trust is low he can weigh priorities, costs and 
feasibilities subjectively, along with other factors, in 
arriving at a final decision on allocation of resources. it 
is important that analysts deal openly and honestly about 
the uncertainties of estimates and quantitative formulas. 
Policymakers can accept these uncertc inties better if they 
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understand the- key principle that a rough approximation 
which accounts for important uncertainties is far more 
useful than a precise answer which ignores them. Whatever 
the actual basis for a decision, we strongly recommend that 
an accurate record be kept of the specific decisions made, 
along with the calculations of priorities, costs and prob- 
abilities of success, so that later comparisons of the 
relative outcomes of quantitatively and qualitatively based 
decisions are possible. 
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